Apr 23
24
Google has updated its Verifying Googlebot and other Google crawlers help document to add a new section describing the three categories or types of crawlers they have. They have their Googlebot crawler, special-case crawlers and user-triggered crawlers.
See the original post here:
Google Documents Its Three Types Of Web Crawlers